The rapid increase in Microprocessor speeds has exceeded the rate of improvement in Dynamic Random Access Memory (DRAM) speeds in recent years. This widening performance gap between processors and memories has created several challenges for computer designers since memory performance can easily limit overall system performance. Specifically, processor performance has been observed to increase at a rate of about 60% yearly, while memory systems lag significantly behind at about 10% yearly improvement. To solve this problem, designers turn to memory performance improvements which ultimately dictate the performance and power consumption of processors. Caching is a common approach used to achieve memory system speed up, by storing data that has been recently used in a local memory. Therefore, using a larger cache could increase the access hit rate, which in turn improves processor speed but this comes with a cost—increased hardware and higher energy consumption.
As a result, there is usually a trade-off between power and performance in memory system design, since not all accessed memory locations can be stored in faster memories such as caches. Current memory systems designed with SRAMs, DRAMs and/or CAMs, have not been able to catch up with processor performance. As a result, larger caches are often employed in memory systems to bridge this memory-processor performance gap. While larger caches offer improved performance, they also increase the power consumed by the processor. An alternative to improved performance is associativity, but this also leads to increased power consumption due to parallel querying of multiple tags. This increasing cache power consumption resulting from the drive for improved performance, cannot be overlooked because caches contribute a significant fraction of the overall power consumed by modern processors. Several authors have concluded that cache/memory systems contribute 30-60% of the total power consumed by processors.
Reducing cache size in an attempt to save power is not a good option either, because it leads to higher miss rates and effectively more power consumption. As a result, attempts have been made to reduce voltages and design lower power circuits to reduce the high proportion of power consumed by caches/memory systems. However, these circuit level techniques have not been very successful; rather, power dissipation levels have steadily increased with each new microprocessor generation, leading to a renewed interest in architectural approaches that reduce the switching capacitive power component of memory systems without sacrificing performance. In an attempt to save power, some researchers have directed their architectural improvements at better performance because of the observation that improved performance (i.e. less misses) usually lead to less power consumption. Others focus on power reduction techniques targeted at specific aspects of the architecture, with some trade off in performance.
Other researchers have focused on the power consumption caused by redundant way accesses in associative caches, and developed techniques for reducing these unnecessary accesses. For example, some approaches have partitioned the data cache into sub-arrays and selected specific group(s) for access based on an algorithm, while using a level zero way cache to store the way information of recent accesses. This concept was extended to Way prediction and selective direct mapping which predicts a way for every new access prior to the cache access, instead of waiting on the tag array lookup and compare step, to provide a specific way number. While these way prediction techniques promise reduced power consumption, they all suffer from some performance degradation due to wrong way predictions which cause access repetitions.